Sheffield MultiMT: Using Object Posterior Predictions for Multimodal Machine Translation
نویسندگان
چکیده
This paper describes the University of Sheffield’s submission to the WMT17 Multimodal Machine Translation shared task. We participated in Task 1 to develop an MT system to translate an image description from English to German and French, given its corresponding image. Our proposed systems are based on the state-of-the-art Neural Machine Translation approach. We investigate the effect of replacing the commonly-used image embeddings with an estimated posterior probability prediction for 1,000 object categories in the images.
منابع مشابه
Sheffield Systems for the English-Romanian WMT Translation Task
This paper provides an overview of the submissions the University of Sheffield for the English-Romanian Translation Task of the ACL 2016 First Conference on Machine Translation (WMT16). The submitted translations were produced with a phrase-based system trained using the Moses toolkit, in two variants: (i) n-best rescoring using additional features from Quality Estimation (primary submission), ...
متن کاملVisually Grounded Word Embeddings and Richer Visual Features for Improving Multimodal Neural Machine Translation
In Multimodal Neural Machine Translation (MNMT), a neural model generates a translated sentence that describes an image, given the image itself and one source descriptions in English. This is considered as the multimodal image caption translation task. The images are processed with Convolutional Neural Network (CNN) to extract visual features exploitable by the translation model. So far, the CN...
متن کاملGhost Image Mapping of Palatal Bone of Maxilla and Nasal Cavity in Panoramic View Using Cranex D Digital Machine
Introdouction: The mapping of ghost images of the maxilla and the nasal cavity, which are complex structures, is very important. The position of objects that create a ghost image can differ when using various devices. The purpose of this investigation was to study the mapping of ghost images of the maxilla and the nasal cavity using a Cranex D digital panoramic machine. Materials and methods: ...
متن کاملMachine Translation - 10: Advanced Neural Machine Translation Architectures
its posterior probability given the training data. This can sometimes be approximated quite well for simple or small models (Xiong et al., 2011; Salakhutdinov and Mnih, 2008), but we would like to approach the performance of the Bayesian gold standard using considerably less computation. We propose to do this by approximating an equally weighted geometric mean of the predictions of an exponenti...
متن کاملUnifying Visual-Semantic Embeddings with Multimodal Neural Language Models
Inspired by recent advances in multimodal learning and machine translation, we introduce an encoder-decoder pipeline that learns (a): a multimodal joint embedding space with images and text and (b): a novel language model for decoding distributed representations from our space. Our pipeline effectively unifies joint image-text embedding models with multimodal neural language models. We introduc...
متن کامل